posted 06-14-2007 12:00 AM            

We went through this a couple of years ago, in response to some bad experiences during 2004. We had a situation in which two examiners wrote one-paragraph QC letters supporting the validity of an examination that was later determined to be a false negative result. At the same time, a few people were quite upset that I didn't agree with the examination, and was attempting to say so. The original examiner was quick to supply the QC letters supporting the examination, but very reluctant to engage in thoughtful discussion about any concerns.

It took three years of committee work to get a QA protocol approved, just a couple of months ago.

Its really a "protocol" for QA activities, and not a QC program. There is no bureaucratic overhead (zero). No QC-leaders, Generals, teachers, preachers, god-squad leaders, or QC police. No random QC. No mandated QC.

It's simply a protocol by which a consumer (PO, therapist, etc.) can request a second opinion on any test, before proceeding with a decision that may affect someone's rights - situations that sometimes end up at the center of legal arguments.

The QA protocol was constructed as a set of requirements for examiner's, consumers/referring agents, and reviewing examiners. It is not intended to engage argument about technique, school, training, and such, but instead it is intended to provide consumers with more information about the strengths or limitations of a particular test, and whether or not their appears to be some serious empirical flaw.

There is no responsible profession anywhere, in which a consumer cannot gain a second opinion or request additional information- or whose professional membership is not expected to accept professional feedback from time to time. The corollary to this is that it is not the role of professional organizations, or professional societies to provide oversight to individual cases - that role includes liability, and belongs to the agencies responsible for the referral, management, and outcome of work related to the individual cases.

So while agencies, departments, and/or beareaus that house polygraph programs can mandate review of individual cases, either periodically, randomly, or routinely, it would create excessive liability for both the organization (composed of distinct individual, program and business entities), examiners (whose work is subject to second-guessing and overturning?), and for the agencies responsible for the individual subjects (that may have made decisions based on tests that were later not supported.) Think about it: if the Utah Poly Assn, were to have mandated QC, and decided that some private practice examiner really blew-it, what responsibility do they have to the client and referring agency? What liability would UPA have if they damage the examiner's business or marketing. That is why this type of QA activity is best designed as a protocol not a program - as a set of guidelines or standards for cooperative professional peer review. It is not intended to be a blind QC program.

Requirements for referring agents are to select a reviewing examiner with the original examiner, and to not automatically assume the reviewing examiner provides more expertise than the original examiner - its just a second opinion. The QA protocol is not intended to become a "program," and is not intended to create liability by searching for problems. It is not intended to discipline or "teach" the examiner. Requirements for the reviewing examiner are to contact the original examiner, provide feedback using a checklist of common features, and support the original examiner's decision unless some empirical flaw is identified that would substantially compromise the validity of the test. This is because original examiner's hand scored results have tended to outperform the results of blind reviewer. The protocol does not include senseless nit-picking about points. Addition requirements are to refrain from make any generalized conclusions or assumptions about examiner competence or any assumption that a single examination could represent the quality of a professional's overall work product - its just one examination. The best examiner in the country could have a bad day, just as the best strike-out pitcher in baseball might give up a few runs.

The protocol simply assures consumers of polygraph tests that examiners will be forthcoming about the strengths and limitations of each examination, and that they won't be surprised in court by the testimony of some opposing counsel's expert reviewer. All tests that go to court are potentially subject to merciless QC, so we might as well learn to do more QA up front before finding ourselves embroiled in un-win-able arguments, and subject to adverse legal precedent. Just look at the Ohio case last year to see a sadly ill-informed argument.

I can obtain the final drafts and send them to you.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 06-14-2007 08:51 AM         

I have a great one that was sent to me by the AAPP. Send me an email at the office today with your contact info and I will email it to you. It is in Word format so you can change it to suite your needs.

posted 06-14-2007 12:38 PM            

QA and QC are terms used mostly interchangeable, but they are somewhat distinct.

QC generally refers to a program - with adminstrative and cost overhead. QA includes a range of activities including requirements for education, training, continuing education, compliance with standard practice, peer review and involvement in quality control activities.

I believe QA is the more appropriate term in the world of private practice - where professional associations have some authoritative influence, but no real authority (which includes liability) over the activities of professionals in private practice.

Most professions would never routinely, pull the files of its professional membership to look for problems - its just plain wrong.

On the other hand, agencies, which might house polygraph programs, already hold liability and can (sometimes do) mandate routine, periodic, or or random review of all or some of their employee's work. That is perfectly reasonable - the agency holds the liability. The important distinction rests in the difference between an "employee" of an agency and a "professional member" of an association.

I our discussion/concussion with examiners over three years of committee work - that is what we learned. All the discussing and cussing was actually quite productive in the end, and we are happy to share our experience.

In the end, all of the examiners who participated in the development of the protocol (not program) were comfortable enough to endorse it. Additionally, probation officers, therapists and state board members and administrators learned a lot too.

I've been hammering on the protocol-not-program point because many of us examiners are used to working in agency and program situations and its important to not remain locked into that mental model when dealing with the realities of private practice and who holds the liability. In many cases it is probation/parole who are ultimately responsible. Therapists (at least in Colorado) have conceded that the supervising officer holds the ultimate liability, and that is reflected in that the PO is designated as the "head" of the community supervision team. We still hear occasional polygraph examiners and therapists whine about equality or inequality in the supervision team, but the reality of the situation is that those same professionals will want no equality when the offender starts acting out against others - they are in fact grateful that the PO is ultimately responsible for the safety and supervision of the offender in the community (with the resources of the county attorneys and presecutor's office to argue their decisions, and the AG's office to argue the reasonableness of their SOP).

Here are some links to a draft from last August. There are some minor edits that occurred recently at the time of approval.

I'll get the updates and upload them shortly. Until then,
http://www.raymondnelson.us/qc/6.180_QC_8-4-06.pdf
http://www.raymondnelson.us/qc/QC_Protocol_Appendix_FINAL_DRAFT_8-4-06.pdf

The appendix contains the checklist, along with a referral/request form and a summary report. The request form is for the PO or therapist. The summary report goes to the subject's file, the checklist goes to the examiner only.

Prior to this we had examiners who refused to allow a second look at their work prior to important decisions. However, the DA's office in some districts will not accept any test that they cannot look at. Years ago I had examiners yell at me when I suggested getting a second opinion before proceeding. More recently we've had at least one examiner become a little escalated in attempt to avoid having to take a look at a test or accept any professional feedback.

We've had some recent experience with the protocol, and it seems to work. We are finding that most examination are just fine - they are informative and we generally agree with each others work product and decision, though we might not want to have to argue every one in court.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)